1. Field
The present technology relates to systems and methods for encoding and decoding audio/video and other digital data. More particularly, the technology relates to computer architecture and operating methods that can enable decoders to decode unsupported formats of audio/video and other multimedia.
2. Description of the Related Art
Digital audio/video and general digital multimedia capabilities can be incorporated into a wide range of devices, including digital televisions, digital direct broadcast systems, wireless communication devices such as radio telephone handsets, wireless broadcast systems, personal digital assistants (PDAs), laptop or desktop computers, digital cameras, digital recording devices, video gaming devices, video game consoles, and the like. Digital devices implement image and video encoding techniques or formats such as JPEG, GIF, RAW, TIFF, PBM, MPEG-2, MPEG-4, and H.264/MPEG-4, Part 10, Advanced Video Coding (AVC), to store, transmit and receive digital images and video efficiently. Digital devices implement audio encoding techniques or formats such as, AAC, MP3, and WAV to store, transmit, and receive digital audio efficiently. Digital devices further implement additional data and graphics encoding techniques or formats such as IGES, 3DT, PS, MNG, ODF and SVG.
Audio, video and other digital data are commonly encoded prior to transmission or storage by an encoder, e.g., a server. The encoding typically consists of operations such as compression or organization into a selected format. The audio, video and other digital data, collectively termed digital multimedia, may be independently stored or provided to a user. Alternatively, the digital multimedia may be embedded in other digital data provided to a user. For instance, an image, video, or animation may be part of an electronic newspaper article, an electronic slideshow, or a technical paper. In either case, the digital multimedia must be decoded prior to display by decoders resident on devices such as mobile devices, DVD players, Blu-Ray players, TV sets, tablets, laptops, computers, or set top boxes. However, a particular decoder may not support decoding of the format used by the encoder. For example, the format used by the encoder may be a legacy format no longer supported by decoders, or may be a new format that the decoder does not support.
Since different decoders may support different formats, traditionally digital multimedia needed to be coded in many different formats to support many different decoders. For example, a user downloading an audio/video file from a server through a network such as the Internet, may have many devices such as a mobile phone, a TV set, a laptop, etc. The downloaded content is traditionally in a single format. However, each of the user's devices may be configured to decode a different format. Accordingly, the user may need to download multiple versions of the audio/video data, each in a different format, for each of the decoders. This leads to bandwidth usage of the network for each version downloaded. Alternatively, the user might download and install a new decoder for each codec type on each device in order to decode the encoded multimedia. However, this solution requires that all legacy formats be supported on all devices. As a third alternative, the user may transcode (decode and re-encode) the digital multimedia received from the downloaded format to each format required for each device. However, this requires computational resources to decode the digital multimedia from the received format and re-encode the digital multimedia into the desired format. Further, this requires memory resources to store each copy of the digital multimedia in each of the desired formats. Additionally, decoding and re-encoding of digital data can lead to loss in quality due to both the loss in precision and the fact that decoding and encoding processes for multimedia data are often lossy as opposed to lossless processes.
One potential technique for avoiding transcoding and to adapt to video content is to provide switches between pre-determined standardized algorithms and tools as suggested in 1997 in Section 2.2.1 of “The MPEG-4 Systems and Description Language: A Way Ahead in Audio Visual Information Representation” by A. Ovaro, et al. As described in Section 2.2.1.3 of this document, the drawbacks include exhaustive specification of all configurations, difficulty of scaling up with increase in available tools, and challenges in anticipating future codec needs.
Similarly, a system for implementing reconfiguration of decoder algorithm elements using flexible or fixed libraries at both the encoder and decoder is proposed in Section 2.2.2 of “The MPEG-4 Systems and Description Language: A Way Ahead in Audio Visual Information Representation” by A. Ovaro, et al. and described in more detail “Whitepaper on Reconfigurable Video Coding (RVC)” ISO/IEC JTCI/SC20/WG11 document N9586 by E. Jang, et al. Information about which tools to select from a decoder library is transmitted either prior to encoded audio/video transmission or is embedded within the compressed audio/video bitstream. Systems for implementing intermittent configuration of algorithms are described in U.S. Pat. No. 5,987,181. Decoding tools or algorithms are selected from local libraries at the decoder through indicators embedded within the compressed bitstream. However, these approaches are limited to specific, pre-determined toolsets and restrict flexibility of systems.
Two systems for partially reconfiguring decoders without the use of local libraries at the decoder are given “Dynamic Replacement of Video Coding Elements” by Bystrom, et al. and “A Syntax for Defining, Communicating, and Implementing Video Decoder Function and Structure” by Kannagara, et al. The first transmits a tool for an inverse transform at the start of an encoded video frame or transmits a binary patch for replacing code in a decoder. The latter transmits encoded algorithms or data with the compressed video and adds the encoded algorithms to the decoder or replaces existing algorithms within the decoder.